Repository navigation
fix: expose DECIMAL precision and scale in result metadata - #563
Merged
Merged
Conversation
vuanhphung
marked this pull request as ready for review
November 14, 2026 05:55
vuanhphung
enabled auto-merge
November 16, 2026 03:59
github-merge-queue
Bot
removed this pull request from the merge queue due to a conflict with the base branch
Nov 16, 2026
Expose the optional precision/scale interface on Thrift and SEA rows using their existing declared schema. Cover populated and empty results, decimal boundaries, manifest qualifiers, and non-decimal columns. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Limit SEA qualifiers to the native Decimal128 schema, keep metadata consistent, and exercise the effective decimal scan options. Include both live tests and DecimalSize parity in nightly coverage. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
vuanhphung
force-pushed
the
fix/issue-500-decimal-metadata
branch
from
November 16, 2026 06:09
ad453f8 to
acf15cf
Compare
vuanhphung
enabled auto-merge
November 16, 2026 06:10
There was a problem hiding this comment.
Verdict: 1 Nit
Looks good, with no blocking issues. Both rows implementations return the declared DECIMAL qualifiers safely: on Thrift they come from the TTypeQualifiers I32 values through the existing bounds-checked getColumnMetadataByIndex, and on the kernel backend they come from the retained Decimal128 schema. Non-decimal and missing-qualifier cases return ok=false, and unit, e2e and parity tests cover populated and zero-row results.
Other findings
- ⚪ Nit —
kernelRows.ColumnTypePrecisionScaleuses the receiver nameresultRows, but every otherkernelRowsmethod usesr. Lint won't catch this because.golangci.yml:59disables ST1016, but usingrwould match the rest of the file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both rows implementations omit Go's optional
driver.RowsColumnTypePrecisionScale, sosql.ColumnType.DecimalSize()reports no DECIMAL metadata. Implement it using existing Thrift qualifiers and the retained SEA Decimal128 schema, including zero-row results. Non-decimals still reportok=false; shared type-name/geo mapping stays untouched.Independent regressions fail before the fix. Verification:
make test(1,477 tests, 9 credential-gated skips),make lint, pure/tagged builds, and root/backend race tests pass. Live populated/empty(10,2)/(38,0)/(38,38), DEC/NUMERIC aliases, and non-decimal controls pass on Thrift (1.49s, native decimals off/on) and SEA (1.35s, decimal-as-float off/on). Full column-metadata parity, now including DecimalSize, passes (1.00s). Both metadata tests and the parity guard are selected by nightly CI. The exact shared DATATYPE-043 test passes live without skipping on Thrift (0.614s) and SEA (0.617s). Shared-warehouse timings measure correctness runs, not performance; metadata-only accessors do not affect request/fetch/conversion paths.Pristine
origin/mainreproduces the same four CloudFetch subtest failures, 21 tagged-lint issues, and three generated-file gofmt findings. Excluding that single CloudFetch parent, the tagged suite passes (1,641 tests, 47 skips). The shared conformance package has an existing Arrow v12/v18 geo compile mismatch on both branches; its unmodified data-types test runs in isolation.Cross-driver state
DatabricksResultSetMetaData.java:600@5ad46ee: declared precision/scale gettersthrift_backend.py:790,kernel/type_mapping.py:133@b4cec221: description qualifiers / Arrow decimalsKernelArrowIpc.ts:261@8ccddd0: manifest qualifiers, Arrow fallbacksrc/reader/mod.rs:951@0b5d0edf: precision/scale metadata and empty schemasFollows declared-schema semantics; unavailable qualifiers return
ok=falserather than an exception because Go's interface has no error return. Unlike Node's text fallback, SEA reads Decimal128 directly: the pinned kernel always reconstructs DECIMAL as Decimal128, independently of scan options.An alternative fix is open in #513. This approach reads the retained schema rather than extending shared type mapping, avoiding the geo workstream, and tests both rows implementations with populated and empty results.
Adversarial review: round 1 converged with no open high findings; seven medium/low findings addressed, and the Thrift conversion-helper refactor deferred to keep the fix scoped.
Fixes #500
This pull request and its description were written by Isaac.